Computer Vision Research Articles

ARTIFICIAL INTELLIGENT SYSTEM FOR AUTOMATIC DEPRESSION LEVEL ANALYSIS THROUGH VISUAL AND VOCAL EXPRESSIONS

Depression is a major mental health disease of human which is rapidly affecting lives worldwide. Early detection and intervention are crucial for effective treatment and management. Depression is regularly identified with thoughts of suicide. Significant depression can bring about a determination of social and physical side effects. It could remember changes in rest, craving, vitality level, concentration, day by day conduct or confidence. In recent years, deep-learned applications concentrated on neural networks have shown superior performance at hand-crafted apps in various areas. This system presents an innovative artificial intelligence (AI) system designed for automatic depression level analysis by analyzing visual and vocal expressions. Leveraging advances in computer vision and natural language processing, the proposed system extracts relevant features from facial expressions and speech patterns to assess depression severity levels accurately. Deep-learned apps that settle the above issues that may precisely assess the degree of voice and face depression. In the proposed method, Convolutionary Neural Networks (CNN) is developed for learning deep-learned features and descriptive raw waveforms for visual expressions. Second. Keywords: Artificial Intelligence, Depression, Vocal, Facial Expressions, Deep Learning

Hybrid deep learning assisted multi classification: Grading of malignant thyroid nodules.

Thyroid nodules are commonly diagnosed with ultrasonography, which includes internal characteristics, varying looks, and hazy boundaries, making it challenging for a clinician to differentiate between malignant and benign forms based only on visual identification. The advancement of AI, particularly DL, provides significant breakthroughs in the domain of medical image identification. Yet, there are certain obstacles to achieving accuracy as well as efficacy in thyroid nodule detection. The thyroid nodules in this study are detected and classified using an inventive hybrid deep learning-assisted multi-classification method. The median blur method is applied in this work to eliminate the salt and pepper noise from the image. Then MPIU-Net-based segmentation is utilized to segment the image. The LGBPNP-based features are retrieved from the segmented image to obtain a single histogram sequence of the LGBP pattern in addition to other features like extraction of multi-texton and LTP-based features. After the feature extraction, the data augmentation process is applied and then the features are fed to the hybrid classification-based nodule classification model that comprises Deep Maxout and CNN, this hybrid classification trains the features and predicts the thyroid nodule. Additionally, the TIRADS score classification is used for the projected malignant thyroid nodule coupled with statistical features collected from the segmented. The DBNAAF with transfer learning model is employed to classify the grading of malignant thyroid nodules, where the weights of the model are learned with transfer learning. The MCC of the Hybrid Model is 0.9445, whereas the DCNN is 0.6858, YOLOV3-DMRF is 0.7229, CNN is 0.7780, DBN is 0.7601, Bi-GRU is 0.7038, Deep Maxout is 0.7528, and RNN is 0.8522, respectively.

Lightweight Deep Learning for Resource-Constrained Environments: A Survey

Over the past decade, the dominance of deep learning has prevailed across various domains of artificial intelligence, including natural language processing, computer vision, and biomedical signal processing. While there have been remarkable improvements in model accuracy, deploying these models on lightweight devices, such as mobile phones and microcontrollers, is constrained by limited resources. In this survey, we provide comprehensive design guidance tailored for these devices, detailing the meticulous design of lightweight models, compression methods, and hardware acceleration strategies. The principal goal of this work is to explore methods and concepts for getting around hardware constraints without compromising the model’s accuracy. Additionally, we explore two notable paths for lightweight deep learning in the future: deployment techniques for TinyML and Large Language Models. Although these paths undoubtedly have potential, they also present significant challenges, encouraging research into unexplored areas.

Object Detection using Convolutional Neural Network

During the last years, a noticeable growth is observed in the field of computer vision research. In computer vision, object detection is a task of classifying and localizing the objects in order to detect the same. The widely used object detection applications are human– computer interaction, video surveillance, satellite imagery, transport system, and activity recognition. In the wider family of deep learning architectures, convolutional neural network (CNN) made up with set of neural network layers is used for visual imagery. Deep CNN architectures exhibit impressive results for detection of objects in digital image. This paper represents a comprehensive review of the recent development in object detection using convolutional neural networks

Enhancing the Energy Efficiency and Robustness of tinyML Computer Vision Using Coarsely-quantized Log-gradient Input Images

This article studies the merits of applying log-gradient input images to convolutional neural networks (CNNs) for tinyML computer vision (CV). We show that log gradients enable: (i) aggressive 1-bit quantization of first-layer inputs, (ii) potential CNN resource reductions, (iii) inherent insensitivity to illumination changes (1.7% accuracy loss across 2 -5 … 2 3 brightness variation vs. up to 10% for JPEG), and (iv) robustness to adversarial attacks (>10% higher accuracy than JPEG-trained models). We establish these results using the PASCAL RAW image dataset and through a combination of experiments using quantization threshold search, neural architecture search, and a fixed three-layer network. The latter reveals that training on log-gradient images leads to higher filter similarity, making the CNN more prunable. The combined benefits of aggressive first-layer quantization, CNN resource reductions, and operation without tight exposure control and image signal processing (ISP) are helpful for pushing tinyML CV toward its ultimate efficiency limits.

TinyM 2 Net-V2: A Compact Low-power Software Hardware Architecture for M ulti m odal Deep Neural Networks

With the evaluation of Artificial Intelligence (AI), there has been a resurgence of interest in how to use AI algorithms on low-power embedded systems to broaden potential use cases of the Internet of Things (IoT). To mimic multimodal human perception, multimodal deep neural networks (M-DNN) have recently become very popular with the classification task due to their impressive performance for computer vision and audio processing tasks. This article presents TinyM 2 Net-V2 —a compact low-power software hardware architecture for m ulti m odal deep neural networks for resource-constrained tiny devices. To compress the models to implement on tiny devices, cyclicly sparsification and hybrid quantization (4-bits weights and 8-bits activations) methods are used. Although model compression techniques are an active research area, we are the first to demonstrate their efficacy for multimodal deep neural networks, using cyclicly sparsification and hybrid quantization of weights/activations. TinyM 2 Net-V2 shows that even a tiny multimodal deep neural network model can improve the classification accuracy more than that of any unimodal counterparts. Parameterized M-DNN model architecture was designed to be evaluated in two different case-studies: vehicle detection from multimodal images and audios and COVID-19 detection from multimodal audio recordings. The most compressed TinyM 2 Net-V2 achieves 92.5% COVID-19 detection accuracy (6.8% improvement from the unimodal full precision model) and 90.6% vehicle classification accuracy (7.7% improvement from the unimodal full precision model). A parameterized and flexible FPGA hardware accelerator was designed as well for TinyM 2 Net-V2 models. To the best of our knowledge, this is the first work accelerating multimodal deep neural network models on low-power Artix-7 FPGA hardware. We achieved energy efficiency of 9.04 GOP/s/W and 15.38 GOP/s/W for case-study 1 and case-study 2, respectively, which is comparable to the state-of-the-art results. Finally, we compared our tiny FPGA hardware implementation results with off-the-shelf resource-constrained devices and showed our implementation is faster and consumed less power compared to the off-the-shelf resource-constrained devices.

Polar-vision1: A Novel Collinearity Equation of Perspective Projection in Polar Coordinate System

Abstract. Progress has been made in the community of photogrammetry and 3d computer vision in addressing the mathematical challenge posed by the collinearity equation. We introduce a new method for establishing the coordinate reference for 2d pixels and 3d landmarks using 'angular coordinates'. The mathematical relationships required for converting 3d landmarks, expressed in angular coordinates, to the camera framework are presented. The landmarks are then projected using perspective projection to obtain 2d pixels represented in angular coordinates. This framework is formally nominated as the 'Polar-vision1', which has been developed and integrated into the commercial software G3D-Cluster. Its application to pinhole camera image processing has demonstrated superior efficiency and admission rates of tie points, as well as reconstruction detail capabilities, compared to OpenMVG, achieving approximately a 1.4× improvement. The project 'Key Technologies and Tool System for Realistic 3D Modeling through Integration of Multi-Source Information in the Space-Air-Ground Domain' was awarded First Prize at the 2023 Surveying Science and Technology Awards, with Polar-vision1 as one of the innovative points.

A modular motion compensation pipeline for prospective respiratory motion correction of multi-nuclear MR spectroscopy

Magnetic resonance (MR) acquisitions of the torso are frequently affected by respiratory motion with detrimental effects on signal quality. The motion of organs inside the body is typically decoupled from surface motion and is best captured using rapid MR imaging (MRI). We propose a pipeline for prospective motion correction of the target organ using MR image navigators providing absolute motion estimates in millimeters. Our method is designed to feature multi-nuclear interleaving for non-proton MR acquisitions and to tolerate local transmit coils with inhomogeneous field and sensitivity distributions. OpenCV object tracking was introduced for rapid estimation of in-plane displacements in 2D MR images. A full three-dimensional translation vector was derived by combining displacements from slices of multiple and arbitrary orientations. The pipeline was implemented on 3 T and 7 T MR scanners and tested in phantoms and volunteers. Fast motion handling was achieved with low-resolution 2D MR image navigators and direct implementation of OpenCV into the MR scanner’s reconstruction pipeline. Motion-phantom measurements demonstrate high tracking precision and accuracy with minor processing latency. The feasibility of the pipeline for reliable in-vivo motion extraction was shown on heart and kidney data. Organ motion was manually assessed by independent operators to quantify tracking performance. Object tracking performed convincingly on 7774 navigator images from phantom scans and different organs in volunteers. In particular the kernelized correlation filter (KCF) achieved similar accuracy (74%) as scored from inter-operator comparison (82%) while processing at a rate of over 100 frames per second. We conclude that fast 2D MR navigator images and computer vision object tracking can be used for accurate and rapid prospective motion correction. This and the modular structure of the pipeline allows for the proposed method to be used in imaging of moving organs and in challenging applications like cardiac magnetic resonance spectroscopy (MRS) or magnetic resonance imaging (MRI) guided radiotherapy.

Virtual Trailroom using Machine Learning

The virtual trail room system receives a real-time video feed from a camera and processes data with the OpenCV computer vision library and Haarcascade classifier. In order to correctly identify faces in the video feed, the Haarcascade classifier is trained on a sizable dataset of human faces.The virtual room system may function independently, which eliminates the need for additional employees and associated expenditures, in contrast to typical security systems that demand the presence of physical security officers.

Virtual Mouse

Now a days the use of hand gesture recognition became popular due to the advance technology of artificial Intelligence (AI). In recent years, there has been a growing interest in developing alternative methods for human-computer interaction (HCI) that go beyond traditional input devices like keyboards and mouse. Hand gesture recognition has emerged as a promising approach, offering intuitive and natural ways for users to interact with digital interfaces. The proposed system utilizes computer vision techniques to detect and track hand gestures in real-time. By mapping specific gestures to mouse control commands, users can navigate and interact with graphical user interfaces (GUIs) without the need for physical input devices. In this paper A hand gesture- controlled virtual mouse system utilizes the AI and Machine learning algorithms to recognize the proper hand gestures and translate them into the mouse movements. The system we are designed that the people who have problems or difficulty using a traditional mouse or keyboard it will be appropriate for them. For this we are using camera that capture images of the user’s hand, which are processed by an AI and ML algorithm to recognize the gesture of hands being made. We trained the system by using a dataset of hand gestures. Keywords — Hand gesture recognition, Machine Learning, Artificial Intelligence, Virtual Mouse, Python, Media Pipe

BACK-to-MOVE: Machine learning and computer vision model automating clinical classification of non-specific low back pain for personalised management.

Low back pain (LBP) is a major global disability contributor with profound health and socio-economic implications. The predominant form is non-specific LBP (NSLBP), lacking treatable pathology. Active physical interventions tailored to individual needs and capabilities are crucial for its management. However, the intricate nature of NSLBP and complexity of clinical classification systems necessitating extensive clinical training, hinder customised treatment access. Recent advancements in machine learning and computer vision demonstrate promise in characterising NSLBP altered movement patters through wearable sensors and optical motion capture. This study aimed to develop and evaluate a machine learning model (i.e., 'BACK-to-MOVE') for NSLBP classification trained with expert clinical classification, spinal motion data from a standard video alongside patient-reported outcome measures (PROMs). Synchronised video and three-dimensional (3D) motion data was collected during forward spinal flexion from 83 NSLBP patients. Two physiotherapists independently classified them as motor control impairment (MCI) or movement impairment (MI), with conflicts resolved by a third expert. The Convolutional Neural Networks (CNNs) architecture, HigherHRNet, was chosen for effective pose estimation from video data. The model was validated against 3D motion data (subset of 62) and trained on the freely available MS-COCO dataset for feature extraction. The Back-to-Move classifier underwent fine-tuning through feed-forward neural networks using labelled examples from the training dataset. Evaluation utilised 5-fold cross-validation to assess accuracy, specificity, sensitivity, and F1 measure. Pose estimation's Mean Square Error of 0.35 degrees against 3D motion data demonstrated strong criterion validity. Back-to-Move proficiently differentiated MI and MCI classes, yielding 93.98% accuracy, 96.49% sensitivity (MI detection), 88.46% specificity (MCI detection), and an F1 measure of .957. Incorporating PROMs curtailed classifier performance (accuracy: 68.67%, sensitivity: 91.23%, specificity: 18.52%, F1: .800). This study is the first to demonstrate automated clinical classification of NSLBP using computer vision and machine learning with standard video data, achieving accuracy comparable to expert consensus. Automated classification of NSLBP based on altered movement patters video-recorded during routine clinical examination could expedite personalised NSLBP rehabilitation management, circumventing existing healthcare constraints. This advancement holds significant promise for patients and healthcare services alike.

Monitoring the spread of a pathogenic insect on vineyards using UAS

Abstract. Globalisation has contributed to rapid economic growth but has also exposed vulnerabilities such as the spread of pests in agriculture. An example is the Popillia Japonica Newman beetle, introduced to Italy in 2014, which has caused significant economic losses, mainly affecting vine cultures. Reliable identification of pests is essential for its management, but it is time-consuming and laborious. This has prompted growing interest in image-based methods, supported by computer vision (CV), which can significantly improve efficiency in insect detection. This study aims to evaluate a CV algorithm's effectiveness in identifying adult specimens of Popillia using Near-Infrared sensors on Uncrewed Aerial Systems (UAS). The project, conducted in two vineyards in northern Italy, intends to establish a replicable and standardised data acquisition protocol for future monitoring activities. Insects detected by the CV-based method are validated by manual counting performed by entomologists. In a GIS environment, prescription maps are generated in near real-time to identify where the vineyard is most affected and to guide the drone spraying treatment only on the areas in which the threshold is exceeded. The study demonstrates effective semi-automated monitoring, with a clear correlation between CV-based and manual insect measurements, as indicated by the Pearson correlation coefficients ranging from 0.89 to 0.96. Although the CV-based method may overestimate insect numbers, it provides valuable insights for targeted pest management interventions and damage assessment. The project outcomes offer a promising approach to safeguarding agriculture against invasive species, enhancing regional economic resilience while minimising the spread of insecticide, the required time, and human interaction with harmful substances.

Efficient Feature Matching and Pose-graph Initialization for SfM

Abstract. Over the last decades, Structure from Motion (or image orientation) has been widely studied in the fields of photogrammetry and computer vision, mainly thanks to its feasibility for dealing with various image datasets (such as crowdsourced or UAV images). However, due to the fact that images are becoming easy to obtain, nowadays, it is challenging to deal with large scale of datasets, wherein the feature matching and pose-graph generation are the key limitations in terms of time efficiency. In this work, we proposed an efficient method to accelerate the generation of correspondences and two-view geometries. Specifically, based on some already estimated two-view geometries, unknown two-view geometries can be computed via A* algorithm. Then, the corresponding feature matching can be perform in a guided way using an epiploar-hash bins that is derived from the estimated two-view epipolar geometries. The experimental results demonstrated that, our method can improve the speed of generating pose-graph by 3–4 times comparing to two popular packages (colmap and OpenMVG) and is also faster than one SOTA method of Barath et al., (2021), yet the results of SfM are typically on par with them and reprojection error of our works are even better.

Dog Breed Identification Using Deep Learning

Deep learning has revolutionized computer vision tasks, including image classification. In this study, we propose a deep learning approach for the identification of dog breeds from images. Leveraging transfer learning with the ResNet50V2 model, our methodology involves preprocessing and augmenting a dataset of dog images to enhance model generalization. Through rigorous experimentation and training, our model achieves competitive accuracy rates in identifying over 60 unique dog breeds. Results indicate the efficacy of our approach in breed classification tasks, with potential applications in pet identification systems and animal welfare initiatives. Keywords: Deep learning, Dog breed identification, Transfer learning, Computer vision.

Identification of Paddy Stages from Images using Deep Learning

Rice, a crucial global staple, is integral to food security. Precise identification of paddy growth stages, booting, heading, anthesis, grain filling, and grain maturity is vital for agricultural decisions. However, a gap exists in recognizing these stages using red-green-blue (RGB) images. This study uses state-of-the-art computer vision and deep learning classification (Convolutional Neural Networks) algorithms to address this gap. Among the studied algorithms, EfficientNet_B0 achieved an impressive 82.8% overall accuracy. Notably, increasing image size from 64X64 pixels to 128X128 pixels significantly enhanced accuracy. A detailed assessment of growth stages revealed varying accuracy levels, with boot leaf being the most accurately detected (95.1%) and anthesis being the most challenging (72.28%). This work significantly advances automated monitoring, empowering researchers in real-time decision-making.

Multi-class identification of tonal contrasts in Chokri using supervised machine learning algorithms

This study examines and explores the effectiveness of various Machine Learning Algorithms (MLAs) in identifying intricate tonal contrasts in Chokri (ISO 639-3), an under-documented and endangered Tibeto-Burman language of the Sino-Tibetan language family spoken in Nagaland, India. Seven different supervised MLAs, viz., [Logistic Regression (LR), Decision Tree (DT), Random Forest (RF), Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Naive Bayes (NB)], and one neural network (NN)-based algorithms [Artificial Neural Network (ANN)] are implemented to explore five-way tonal contrasts in Chokri. Acoustic correlates of tonal contrasts, encompassing fundamental frequency fluctuations, viz., f0 height and f0 direction, are examined. Contrary to the prevailing notion of NN supremacy, this study underscores the impressive accuracy achieved by the RF. Additionally, it reveals that combining f0 height and directionality enhances tonal contrast recognition for female speakers, while f0 directionality alone suffices for male speakers. The findings demonstrate MLAs’ potential to attain accuracy rates of 84–87% for females and 95–97% for males, showcasing their applicability in deciphering the intricate tonal systems of Chokri. The proposed methodology can be extended to predict multi-class problems in diverse fields such as image processing, speech classification, medical diagnosis, computer vision, and social network analysis.

Transformer-based intelligent fault diagnosis methods of mechanical equipment: A survey

Abstract Transformer is extensively employed in natural language processing, and computer vision (CV), with the self-attention structure. Due to its outstanding long-range dependency modeling and parallel computing capability, some leading researchers have recently attempted to apply Transformer to intelligent fault diagnosis tasks for mechanical equipment, and have achieved remarkable results. Physical phenomena such as changes in vibration, sound, and heat play a crucial role in the research of mechanical equipment fault diagnosis, which directly reflects the operational status and potential faults of mechanical equipment. Currently, intelligent fault diagnosis of mechanical equipment based on monitoring signals such as vibration, sound, and temperature using Transformer-based models remains a popular research topic. While some review literature has explored the related principles and application scenarios of Transformer, there is still a lack of research on its application in intelligent fault diagnosis tasks for mechanical equipment. Therefore, this work begins by examining the current research status of fault diagnosis methods for mechanical equipment. This study first provides a brief overview of the development history of Transformer, outlines its basic structure and principles, and analyzes the characteristics and advantages of its model structure. Next it focuses on three model variants of Transformer that have generated a significant impact in the field of CV. Following that, the research progress and current challenges of Transformer-based intelligent fault diagnosis methods for mechanical equipment are discussed. Finally, the future development direction of Transformer in the field of mechanical equipment fault diagnosis is proposed.

Visual Reinforcement Learning for Dynamic Object Detection

Abstract. Object detection is a widely studied task in computer vision. Current methods often focus on images captured from appropriate viewpoints. However, there is a large disparity between objects observed from different viewpoints in the real world. Dynamic Object Detection (DOD) method automatically adjusts the camera viewpoint in a visual scene to sequentially find optimal viewpoints. Currently, the DOD tasks are usually modeled as a sequential decision-making problem and solved using reinforcement learning methods. Existing approaches face challenges with sparse rewards and training instability. To tackle these issues, we proposed a single-step reward function and a lightweight network, respectively. The single-step reward function, which provides timely feedback, gives an efficient training process for DOD tasks. The lightweight network with few parameters can ensure the stability of the training process. To evaluate the effectiveness of our method, we developed a simulation dataset based on UE4, which consists of 1800 training images and 450 testing images. The dataset includes five object categories: vans, cars, trailers, box trucks and SUVs. Experiments demonstrate that our method outperforms SOTA object detectors on our simulation dataset. Specifically, the average precisions(APs) are improved from 89.1% to 96.0% when using the YOLOv8 object detector.

Assessing the reliability of a computer-vision tool for curing postural deformities

Postural deformities, such as rounded shoulders, kyphosis, and lordosis, are a prevalent issue affecting individuals of all ages. These deformities can lead to pain, discomfort, and reduced mobility, significantly impacting one's quality of life. Traditional methods for addressing postural deformities include physiotherapy, exercise, and postural braces, which can be time-consuming, expensive, and difficult to adhere to. To solve these problems computer vision technology has emerged as a promising tool for evaluating and correcting postural deformities. Computer vision algorithms can analyze images and videos of an individual's posture, identifying postural abnormalities and providing real-time feedback. The fusion of computer vision technology with AI-driven analytics not only revolutionizes postural correction methodologies but also paves the way for comprehensive, accessible, and efficient solutions to address the spectrum of postural deformities. As this technology continues to evolve, it holds immense potential in reshaping the landscape of posture-related healthcare, offering tailored, accessible, and effective interventions to enhance individuals' postural well-being. Keywords: Postural deformities, AI-driven analytics, Computer vision technology, Dashboard

Multi-class waste segregation using computer vision and robotic arm

Waste segregation is an essential aspect of a smoothly functioning waste management system. Usually, various recyclable waste types are disposed of together at the source, and this brings in the necessity to segregate them into their categories. Dry waste needs to be separated into its own categories to ensure that the proper procedures are implemented to treat and process it, which leads to an overall increased recycling rate and reduced landfill impact. Paper, plastics, metals, and glass are just a few examples of the many dry waste materials that can be recycled or recovered to create new goods or energy. Over the past years, much research has been conducted to devise effective and productive ways to achieve proper segregation for the waste that is being produced at an ever-increasing rate. This article introduces a multi-class garbage segregation system employing the YOLOv5 object detection model. Our final prototype demonstrates the capability of classifying dry waste categories and segregating them into their respective bins using a 3D-printed robotic arm. Within our controlled test environment, the system correctly segregated waste classes, mainly paper, plastic, metal, and glass, eight out of 10 times successfully. By integrating the principles of artificial intelligence and robotics, our approach simplifies and optimizes the traditional waste segregation process.